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Executive Summary 

In 1997, Indiana’s Early Literacy Intervention Grant Program (ELIGP, also referred to 
as EIGP) was initiated by Dr. Snellen Reed, Superintendent of Public Instruction, with 
the support of the Indiana General Assembly. The grant provided annual funding for 
schools to implement Reading Recovery, a highly regarded early literacy program, or 
other literacy interventions. Prom its inception, approximately $31.4 million has been 
distributed through the grant program. 

Since 1997, the Center for Evaluation and Education Policy (CEEP, formerly the Indi- 
ana Education Policy Center) has conducted a series of evaluations on ELIGP imple- 
mentation. This report draws from eleven previous reports in order to describe and 
provide evidence of the effectiveness of the ELIGP program between 1997 and 2004. 
bindings are based on a variety of data, including records provided by the Indiana 
Department of Education (IDOE), surveys collected from principals and teachers at 
funded and comparison schools, observational and interview data collected during site 
visits, and ISTEP-i- results. 
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Findings indicate that ELIGP has had a considerable impact on literacy education in 
Indiana, serving more than 10,000 students per year and over 700 schools since 1997. 
While analysis across the eight years of the grant is complex, due in part to the vari- 
ability in interventions chosen each year of the grant, data generally links funding to 
improvements in areas of teacher professional development, lower grade retention, 
decreased special education referrals, and increased student achievement. 

Among Themes Consistently Cited in CEEP Reports: 

• ELIGP funds are apparently being awarded to the intended recipients — 
students at high risk for literacy difficulties. And most of the funded 
interventions, including Reading Recovery, appear particularly effec- 
tive among this target group. 

• Programs appear to do best given time to develop, as demonstrated in 
evaluations comparing success rates between first-year programs and 
those with extended funding. 

• Programs associated with success do not necessarily represent extreme 
departures from standard literacy instruction in Indiana schools. 

Rather, those programs tend to reflect a balanced approach to literacy 
education including a comprehensive set of features. Contextual, holis- 
tic approaches, as well as more direct, phonics-oriented tactics, are 
both found in the most successful programs funded by ELIGP, and the 
combination appears to be associated with desired student outcomes. 



II of IV 



Center for Evaluation and Education Policy 



Based on these conclusions and drawing from seven years of ELIGP data analysis, the 
following recommendations are made to the IDOE: 

Research and Evaluation 

• Continue to conduct an annual survey of EEIGP and comparison 
schools with the sustained funding in the 2005-07 state budget. Experi- 
mental and/or multi-year designs should be considered, to allow for the 
most rigorous evaluation of program outcomes. 

• Continue to identify research-based programs and provide schools 
with information and assistance to guide their literacy intervention 
choices and select the most appropriate program for their school con- 
text. 

• Encourage schools to review and refine their early reading and literacy 
programs through systematic evaluations. 

Funding for Early Literacy Initiatives 

• Eunding for packaged programs should be targeted on interventions 
with proven records of success (i.e., evidence-based best practices), 
especially interventions that focus on reading improvements in Grades 
1-3. Of the EEIGP interventions assessed by CEEP, Eiteracy Collabo- 
rative appears especially successful, demonstrating the most consistent 
positive results. Reading Recovery, Success for All, and Eour Blocks 
all demonstrated favorable results overall. Early Success shows prom- 
ise, based on limited data collected on that intervention, and merits fur- 
ther attention. The Eirst Steps and Waterford programs both received 
mixed results and should be examined further. 
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• Funding for locally designed interventions should be considered care- 
fully, based on each school’s previous levels of success implementing 
literacy reforms. Ongoing funding for these programs should be deter- 
mined by the effectiveness and sustainability of each program. 

• Encourage schools serving low-income students to participate in the 
ELIGP program. Technical support for planning literacy interventions 
and applying for funding should be made available to low-income 
schools that have not applied previously. 

Providing Support to Indiana Schoois 

• Increase emphasis on ongoing professional development in early read- 
ing and literacy for elementary teachers. 

• Utilize local university support to enhance the implementation of early 
literacy intervention models. The IDOE should facilitate or encourage 
closer collaboration between schools and universities concerning 
ongoing professional development and evaluation that would benefit 
individual schools and the EEIGP grant program as a whole. 

• The IDOE should consider resuming efforts to provide grant work- 
shops and expert guidance on reading research and literacy program 
implementation. 
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Indiana’s Early Literacy 
Intervention Grant Program 
1997-2004 

Introduction 

The Early Literacy Intervention Grant Program (ELIGP) was initiated by the Indiana 
Department of Education in 1997, under the leadership of Dr. Suellen Reed, Superinten- 
dent of Public Instruction, and with the support of the Indiana General Assembly. The 
ELIGP was designed to support schools in their efforts to develop early literacy programs 
that meet the needs of Hoosier students at risk of school failure. That year, ELIGP pro- 
vided funds for new projects that served 262 elementary schools in approximately 107 
Indiana school corporations. Since then, annual funding has continued with the goal of 
increasing the literacy skills of students in Grades K-3 who are at risk for school failure. 

Since the 1997-1998 funding year, the Center for Evaluation and Education Policy 
(CEEP - formerly the Indiana Education Policy Center) has conducted annual evaluations 
of the ELIGP. The evaluations have used a variety of methods to document implementa- 
tion and outcomes associated with the grant. Specific foci of program reports have 
included descriptive analyses of funded programs, and statistical analyses examining the 
relationship between funding and student achievement, special education referrals, and 
grade retention in ELIGP-funded and comparison schools. 

Data collection and analysis methods have included principal and teacher surveys; finan- 
cial analysis; site visits and case studies; content analysis of program components; and 
analysis of retention rates, special education referral rates, and ISTEP-i- results. The pur- 
pose of the current report is to synthesize 11 previous CEEP project reports in order to 
provide a comprehensive overview of ELIGP implementation and effectiveness. 
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Information presented in this report draws from CEEP involvement across eight years 
of the grant program and includes data on financial aspects of the program, its impact 
on schools, and select student outcomes. Results regarding specific initiatives are 
detailed briefly, and conclusions are drawn regarding benefits generated by the grant 
and which students appear to gain most from the initiatives. Taken together, these 
cumulative findings lead to a number of implications and recommendations which 
conclude the report. 

This report focuses on interventions in Grades 1-3, because that has been the primary 
focus of the EEIGP grant across its life. Although some preschool (Even Start) and 
kindergarten programs were the recipients of grant funds during some of these years, 
those data are limited and insufficient to support the type of summative findings that 
this report seeks to generate. Table 1 lists major reports produced by the CEEP that 
were referenced in the current evaluation, the components of those studies, and the 
outcomes examined in each. 



TABLE 1. Table 1 : Prior ELIGP Studies Completed by CEEP and Referenced in the Current Report. 



Publication 

Year 


Research Approach 


Data 

Timeframe 


Outcomes of Interest 


1999 


Survey of principals 
in grant and compari- 
son schoois, site vis- 
its, case studies 


1997-1998 


* Retention 

* Referrais 

* Instructionai components 


1999 


Resource Guide 


N/A 


• Deveioping research-based 
programs 


2000 


Survey of principais 
in grant and compari- 
son schoois, site vis- 
its, case studies 


1998-1999 


* Program features 

* Retention 

* Referrais 

* Instructionai components 


2000 


Survey of principais 
in grant and compari- 
son schoois 


1999-2000 


* Program features 

* Retention 

* Referrais 

* Instructionai components 
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TABLE 1. Table 1 : Prior ELIGP Studies Completed by CEEP and Referenced In the Current Report. 



Publication 

Year 


Research Approach 


Data 

Timeframe 


Outcomes of Interest 


2000 


Impact study combin- 
ing data from first 
three years of ELIGP 
grant 


1997-2000 


• Program features 

• Retention 

• Referrals 

• Instructional components 

• ISTEP-i- results 


2000 


Analysis of funding 
across first 3 years of 
ELIGP grant 


1997-2000 


• Amount and percentage of 
funding for Reading Recovery 
and OELI projects 

• Trends in funding 


2001 


Descriptive study of 
achievement gains in 
two ELIGP grant 
schools 


2000-2001 


• Pre- and post-student scores 
on Basic Academic Skills 
Sample (BASS) 


2001 


Survey of principals 
and teachers in grant 
and comparison 
schools 


2000-2001 


• Program features 

• Retention 

• Instructional components 

• ISTEP-i- results 


2002 


Resource Guide 


N/A 


• Methods for selecting and 
assessing reading programs 


2003 


Online survey of prin- 
cipals and teachers in 
grant and comparison 
schools, site visits, 
case studies 


2001-2002 


• Program features 

• Retention 

• Instructional components 

• ISTEP-I- results 


2004 


Site visits and case 
studies comparing 
Four Blocks and 
Waterford programs 


2002-2003 


• Application of instructional 
components in classroom 

• Teacher opinion 

• Student activities 

• ISTEP-I- results 



Grant Details, 1997 - 2004 

Since the program’s inception, ELIGP funding has been distributed to approximately 
700 Indiana schools to support Reading Recovery, the Waterford Program, and other 
literacy initiatives. While the level of state funding has remained relatively constant, 
excepting a 7 percent cut in 2002 under the Deficit Management Plan, the total amount 
of funds given and the proportion dispersed to intervention types has varied from year 
to year. CEEP reports have provided descriptive data documenting distribution of 
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funds for each year of the grant, with the exception of 2002-2003, for which reliable 
data were not available from IDOE. 

Table 2 summarizes ELIGP funding information across the seven years of available data. 
These data indicate that a total of 674 different schools from 203 school corporations 
participated in at least one of the grant years. This total excludes schools from 2002- 
2003, for which no reliable data were available; using a conservative estimate of partici- 
pants for that year, the total would be easily over 700 different schools across the eight 
years. 



TABLE 2. Table 2: ELIGP Funding 1 997-2004 





97-98 


98-99 


99-00 


00-01 


01-02 


02-03* 


03-04 


Reading 

Recovery 

trainer 


$596,482 


$105,000 


$177,000 


$183,000 


not available 




not available 


Reading 

Recovery 


$1,104,000 


$1,554,000 


$1,197,000 


$1,027,500 


$1,072,500 




$441 ,496 


OELI* 


$1,662,335 


$1,724,220 


$1,706,603 


$1,776,104 


$2,177,005 




$3,091,060 


TOTAL 


$3,362,817 


$3,383,220 


$3,080,603 


$2,986,604 


$3,249,505 




$3,532,556 


# of 

Reading 

Recovery 

schools 


140 


173 


126 


103 


113 




133 


# of OELI 
schools 


142 


131 


76 


79 


87 




78 


# of schools 


282 


304 


202 


182 


200 




211 


Est. # of 
students 
served 


9,685 


19,396 


10,860 


14,487 


not available 




not available 



* Data for 2002-2003 were not available 



Other Early Literacy Intervention 



1997-1998 

The first study completed by CEEP, titled “Indiana’s Early Eiteracy Intervention 
Impact Study for 1997-1998,” reported that $1.7 million went to Reading Recovery, 
training 184 teachers and 10 new Reading Recovery teacher trainers (teacher leaders) 
in 70 corporations. Eunds directly benefited 140 schools. Also, 54 school corporations 
received funding for a total of 63 Other Early Eiteracy Intervention (OEEI) projects, 
including Eour Blocks, Success for All, and others. Almost all school corporations that 
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applied for grants were funded for at least half of the proposal budget. In all, an esti- 
mated 9,685 students were served by ELIGP in 1997-1998. The total amount spent in 
direct intervention funding was $2,766,335. 

1998 - 1999 

The following year witnessed an increase in funding, with $3,383,220 in grants 
awarded to schools and corporations (source: “Indiana’s Early Eiteracy Intervention 
Impact Study for 1998-1999”). Purdue University received $105,000 of those funds 
for instruction of 10 Reading Recovery trainers (teacher leaders) in seven corpora- 
tions. In addition, $1,554,000 went toward Reading Recovery programs in 79 corpora- 
tions and 173 schools. In all, an estimated 2,296 students were served by Reading 
Recovery in 1998-1999 through the support of EEIGP grant funds. The cost was esti- 
mated at approximately $677 per student. Also, OEEI programs were supported in 131 
schools, with $1,724,220 in grants. Approximately 19,396 students were served with 
these grants, at a cost of around $89 per student. 

1999 - 2000 

In the 1999-2000 year, $3,080,603 in EEIGP funds was distributed (source: “Indiana’s 
Early Eiteracy Intervention Impact Study for 1999-2000”). Purdue University received 
$177,000 of those funds for the instruction of three Reading Recovery trainers (teacher 
leaders) serving in three corporations. This cost, for training new Reading Recovery 
teacher leaders, includes replacement salaries (up to $35,000) and living expenses for 
trainees in addition to direct training costs. The bulk of Reading Recovery funds, 
$1,197,000, supported Reading Recovery programs in 61 corporations and 126 
schools. An estimated 1,823 students received Reading Recovery through the support 
of the EEIGP program, at a cost of approximately $657 per student. OEEI programs 
were supported in 76 schools with $1,706,603 in EEIGP grants, serving approximately 
10,860 students. 
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1997-2000 Funding 

The distribution of funding across intervention types during this first three years of 
ELIGP funding was among data presented in a 2001 report titled “Indiana’s Early Eit- 
eracy Intervention Grant Program Budget Report 1997-2000.” According to the 
report, the percent of funds allotted to Reading Recovery projects increased from 
1997-1998 to 1998-1999 and then decreased from 1998-1999 to 1999-2000. The 
opposite trend was found for OEEI projects. The percent of total funding allocated to 
OEEI projects decreased from 1997-1998 and then increased from 1998-2000. 

2000-2001 

In 2000-2001, $2,986,604 in EEIGP funds was awarded for literacy interventions 
(source: “Indiana’s Early Intervention Grant Program Impact Study for 2000-2001”). 
Of those funds, $183,000 was awarded to Purdue University for training of three Read- 
ing Recovery trainers (teacher leaders) to serve in three school corporations. A total of 
$1,027,500 was granted to support Reading Recovery interventions in 53 corporations 
and 103 schools. The estimated number of students served by Reading Recovery teach- 
ers funded by EEIGP in 2000-2001 was 1,816. Therefore, excluding funding for Read- 
ing Recovery trainers, the state cost for Reading Recovery was approximately $566 per 
student that year. Other OEEI projects were granted $1,776,104, with those funds going 
to 79 schools within 52 corporations and serving an estimated 12,671 students. The cost 
for these interventions was approximately $140 per student. 

2001-2002 

EEIGP grant funds in 2001-2002 totaled $3,249,505 (source: “Indiana’s Early Inter- 
vention Grant Program Impact Study for 2001-2002”). Of these funds, $1,072,500 
supported Reading Recovery interventions in 61 corporations and 113 schools. The 
remaining $2,177,005 supported other interventions (OEEIs) in 87 schools within 49 
Indiana school corporations. 
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2002 - 2003 / 2003-2004 

Reliable funding information for 2002-2003 was not available from IDOE, and there- 
fore is not included in this report. In 2003-2004, a total of $3,532,556 in grants was 
awarded. Of those funds, $441,496 was allotted to Reading Recovery programs at 133 
schools, and $3,091,060 supported OELI programs at 78 schools. 

Table 3 reports the number of grants received by schools across seven years of avail- 
able data. As shown, a total of 674 different schools were funded between 1997 and 
2004 (excluding 2002-2003), and slightly more than one half (55 percent) of partici- 
pating schools received more than one year of funding. 



TABLE 3. Table 3: Number of Grants Received per School Participating in ELIGP Funding 1997-2004. 




Years of Funding Received 
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Impact on Classroom Instruction 

Because the design of the evaluation studies has often included a comprehensive anal- 
ysis of both comparison schools and grant schools, the data allow inferences regarding 
the general literacy environment found throughout the state, as well as the impact of 
grant funding. This section, based primarily on studies comparing funded and compar- 
ison schools, discusses how the grant appears to effect change within schools. The 
focus in this section is on programmatic changes that appear related to grant funding, 
as opposed to student outcomes, which are addressed in subsequent sections. The sec- 
tion first discusses the overall status of literacy education as found in the studies, and 
then focuses on how the grant schools differed from comparison schools. 

Climate of Literacy Instruction in Indiana Schools 

Since the first data were collected, CEEP reports have noted aspects of grant initiatives 
that appear to be consistent with the prevailing climate of literacy instruction in Indi- 
ana schools. The 1997-1998 report found that elementary schools in the state used a 
balanced approach to early reading and literacy, in terms of organization, instruction, 
and classroom philosophy. That report did not find initiative features to be drastic 
departures from what schools were already attempting to provide, but concluded that 
the new interventions were implemented in a state- wide context of instruction that was 
largely consistent with the interventions. Eor example, Indiana schools in both EEIGP 
and comparison groups typically used independent reading and small groups as daily 
strategies for teaching early reading. The small groups were found to allow for system- 
atic instruction, while independent reading promoted individual student interests and 
accommodated diverse learning styles. Phonics and reading drills as well as creative 
writing and emergent spelling were common as well. These findings were taken to 
demonstrate that both systematic and holistic philosophies and approaches were being 
used in the Indiana schools studied. Other features that were valued in both funded and 
non-funded Grade 1-3 classrooms included trade books, a holistic literature-rich 
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approach, basal readers and Big Books, and systematic student performance evalua- 
tion. 

Likewise, the 2000-2001 report noted that many Reading Recovery strategies were 
used frequently or very frequently by teachers in all types of schools, whether grant- 
funded or not. This was interpreted as evidence of a high degree of congruence 
between Reading Recovery methods and strategies used by regular classroom teachers. 



Features of Funded ELIGP Interventions 

Thus, while the instructional approaches of successful ELIGP interventions may not 
be radically different from those already found in Indiana schools, the data have 
enabled us to identify where funded and non-funded schools differ with respect to spe- 
cific features and program emphasis. CEEP reports have been able to identify the 
extent to which individual features vary among interventions and comparison schools, 
as described below. 

The 1997-1998 report found that the greatest change seen within EEIGP-funded 
schools was an increase in the use of systematic formative evaluation, trade books, 
emergent spelling, creative writing, drama, and paired reading. The report concluded, 
in part, that the funding led to an emphasis on ongoing evaluation of the progress of 
each student and individualizing of the curriculum. Among other findings, the 1997- 
1998 study reported that EEIGP schools were more likely than comparison schools to 
organize their classrooms with alternatives to whole class instruction, including ability 
grouping, child-initiated learning centers, one-on-one tutoring, pullout instruction, 
small group interaction, and cooperative learning. EEIGP schools reported more fre- 
quent use of trade books. Big Books, basal readers, systematic formative evaluation 
(e.g., performance assessment such as portfolios or running records), phonics, reading 
drills, creative writing, and drama. 
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The 1998-1999 report found that frequency of instructional features was relatively 
similar between comparison and ELIGP schools. A notable exception included a find- 
ing that creative/essay writing was used more frequently in OELI schools than in com- 
parison schools. Also, OEEI schools reported that they were less likely to use 
worksheets/workbooks components than comparison schools. That report suggested 
that greater reliance on worksheets could possibly be explained by the capacity to use 
such sheets without training in literacy methods such as those in the grant program. 

The 1998-1999 report also examined the degree to which participants’ philosophies of 
their early literacy program leaned toward holistic and reductionist or skills-based 
approaches (Student-Directed versus Teacher-Directed Instruction, Prescribed Sys- 
tematic versus Child-Centered/Developmental Curriculum, Code/Phoneme Empha- 
sized versus Meaning/Comprehension, and Code/Phoneme Taught Within versus 
Outside of Context). While, as reported above, schools were likely to include a bal- 
anced approach to literacy instruction, OEEI schools were found to be more Student- 
Directed and to have more Child-Centered/Developmental curricula than comparison 
schools. As an example, OEEI schools reported a significantly greater use of learning 
centers than did comparison schools. This type of child-initiated means of instruction 
allows for individualized instruction reflecting student choice and engagement. There- 
fore, EEIGP funding appears to be related to increased use of child-centered instruc- 
tion occurring in a natural, rather than isolated, context. 

The 1999-2000 report revealed that the EEIGP funding resulted in more time spent in 
activities where students were either reading independently or to a partner than 
occurred in comparison schools. Students were also more likely to be engaged in cre- 
ative/essay writing and less likely to use workbooks or worksheets. Again, EEIGP- 
funded schools reported greater increases in the frequency of the creative/essay writ- 
ing component, as well as emergent spelling and reading drills, than comparison 
schools. EEIGP schools also had classroom organizational structures that foster more 
individualized and intensive instruction including ability grouping, small groups, 
child-initiated learning centers, and cooperative learning. 



10 of 48 



Center for Evaluation and Education Policy 



The 2000-2001 report identified three areas which were more prevalent among Read- 
ing Recovery programs than other schools. First, pull-out instruction was found most 
frequently in Reading Recovery schools, which is not a surprising finding, given that 
this is a core feature of the program. Ongoing written observations were found more 
frequently in schools with Reading Recovery than other schools. Finally, reading spe- 
cialists were most likely to be employed in Reading Recovery schools. Reading 
Recovery teachers were found in 69% of all classrooms in Reading Recovery schools 
and, similarly, 66% of classrooms in OELI schools had Reading Recovery teachers. In 
contrast, only 36% of classrooms in comparison schools had Reading Recovery teach- 
ers. The fact that findings did not demonstrate extreme departures from instructional 
norms at Reading Recovery schools was in line with consistent findings across study 
years. As in earlier studies, the 2001 report concluded that features associated with 
Reading Recovery were common in other reforms and in regular Grade 1 classrooms 
and that there is “a high degree of compatibility between the features of Reading 
Recovery and the Day-to-day practices of Grade 1 teachers in Indiana elementary 
schools” (p. 50). This evident compatibility between the features of Reading Recovery 
and the practices of regular Indiana elementary teachers was interpreted to indicate “a 
high degree of consonance in the structure and practice of reading and language arts 
instruction across elementary schools in Indiana” (p. 36). Therefore, it appears to be 
the amount of attention placed on various aspects of literacy instruction among ELIGP 
programs that may contribute to their apparent success, rather than the novelty of those 
program features. 



Professional Development and 
Collaboration 

One common finding in previous reports regards increased levels of professional 
development in funded schools. The 1997-1998 report found that EEIGP schools had a 
higher level of expertise available than comparison schools. Both Reading Recovery 
and OEEI schools were more likely to have certified specialists as teachers or to pro- 
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vide training to other teachers. In addition, both Reading Recovery and OELI program 
participants reported greater opportunities for networking and collaboration than com- 
parison schools. The finding that professional development was used more extensively 
in ELIGP than comparison schools was considered evidence that the grant funding 
made that outcome possible. 

Again, in 1998-1999, a CEEP report found that a greater percentage of funded schools 
than comparison schools reported an increase in professional development features in 
terms of increases in certified training and specialists, in-service workshops, and 
opportunities for networking and collaboration. 

bindings were replicated in the 1999-2000 report, which found that funded schools 
reported significantly more professional development than did comparison schools. 
OEEI schools reported a greater frequency of literacy related in-service workshops 
than comparison schools. Both Reading Recovery and OEEI schools reported a greater 
use of certified specialists and the opportunity for networking and collaboration 
among professionals than comparison schools. 

In addition to increased professional development and collaboration opportunities 
apparently made possible by EEIGP funds, some evidence was found that participat- 
ing in the program itself was associated with increased levels of collaboration among 
teaching faculty. The 2001-2002 report found that although Eour Blocks does not 
include a specific emphasis on professional development, those schools implementing 
that program had higher levels of collaboration than comparison schools (though less 
than other funded schools). The report concluded that the emphasis on professional 
development in the IDOE grant process may help explain the generally higher levels 
of collaboration found in funded versus comparison schools. It would appear that sim- 
ply participating in the EEIGP grant is associated with increased levels of collabora- 
tion, whether or not a given funded school’s literacy intervention emphasizes 
collaboration. 
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Parental Involvement 



Parent involvement is often mentioned as an important part of education reform and 
increasing student achievement. In general, ELIGP findings support that the program 
was associated with increases in certain types of parent involvement. In 1997-1998, 
ELIGP schools reported a greater degree of parent involvement in their programs than 
comparison schools. Reading Recovery and OELI schools were more likely to distrib- 
ute books to households with a low number of reading materials, include literacy 
instruction for parents, have parent/child reading programs, and utilize parent volun- 
teers more frequently. The 1997-1998 report concluded that “ELIGP funding is serv- 
ing to support parent involvement in school programs and maximizing student 
learning potential by supporting literacy practices at home” (p. v). The 1998-1999 
report found four aspects of parent involvement that were indicated to increase signif- 
icantly in ELIGP schools compared with comparison schools: book distribution, fam- 
ily literacy instruction, paired reading (parent/child), and parent/teacher conferences. 
Results for the following year (1999-2000) also demonstrated a significantly greater 
use of book distribution, family literacy instruction, and paired reading (parent/child) 
in OELI than comparison schools. In contrast to the prior year, that 1999-2000 study 
found higher levels of parent volunteers among comparison schools than funded 
schools. Thus, ELIGP funding appears to be consistently associated with several fea- 
tures of parent/family involvement including distribution of books to student house- 
holds, instruction addressing literacy needs of the entire family, and promotion of 
parent/child reading, bindings regarding the use of parent volunteers in ELIGP schools 
were inconsistent and may be an area for future research. 



Student Outcomes - Retention 

Retention rates — the percentage of students retained at current grade level — ^have been 
used in CEEP studies as a measure of school performance. As noted in the 1997-1998 
report, “Researchers have viewed such measures as concrete indicators of whether a 
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child has performed acceptably (Bronfenbrenner, 1979)” (p. 82). While retention may 
reflect a number of student characteristics (i.e., social immaturity, adjustment difficul- 
ties), most children are retained for academic failure. Thus, retention can also be an 
indicator of the inflexibility of the instructional environment and instructional intoler- 
ance to individual student differences. A reduction in grade retention has been used in 
CEEP studies as one potential indicator that EEIGP-funded programs contribute to 
positive literacy outcomes for students at risk. Retention is the worst intervention for 
academic problems and rarely suitable for children struggling in school (Darling- 
Hammond, 1998). 

EEIGP data collected by CEEP have consistently provided at least marginal support 
that funding is associated with reduced retention, although findings have often failed 
to meet statistical requirements for significance. The 1997-1998 report concluded that 
funded schools appeared generally less likely to retain students than comparison 
schools, although the difference did not reach statistical significance. Reading Recov- 
ery schools were found to have higher retention rates than comparison schools, while 
OEEI schools had lower retention rates than comparison schools. These findings could 
reflect actual differences between the schools, but they could represent findings attrib- 
utable to chance, since statistical significance was not reached. 

A study reporting on retention rates for three years, 1998-2000, found that OEEI 
schools’ retention rates decreased across those years, although again, those rates did 
not vary to a degree that was statistically significant. Retention rates in Reading 
Recovery schools increased in 1998-1999, as reported above, but were found to be rel- 
atively stable in the 1999-2000 report. 

The 2000-2001 report found lower grade-level retention rates in Reading Recovery- 
funded schools than in comparison schools with comparable poverty levels. Schools 
with Reading Recovery funding retained 1.14% of their students, compared to 1.37% 
in comparison schools. Most OEEI (class-wide) intervention models also resulted in 
lower rates of retention, even more substantial than Reading Recovery findings. Those 
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Grade 1-3 programs had lower retention rates (1.06%) than either comparison schools 
(1.37%) or Reading Recovery-funded schools (1.14%). 

The 2001-2002 report found the average retention rate for all ELIGP- funded schools 
to be .83%, compared to a higher retention rate (1.27%) for comparison schools. Mean 
retention in Reading Recovery schools was .57%, compared to 1.27% in comparison 
schools. The difference was most pronounced in the highest poverty quartile, .65% 
(Reading Recovery) vs. 2.96% (comparison), as discussed in a following section of 
this report. 

A further breakdown of the specific programs in 2001-2002 found that all interven- 
tions except the Waterford program were associated with lower retention rates than 
comparison schools. In particular, the Reading Recovery and Four Blocks interven- 
tions were pointed out as having substantially lower retention rates. When controlling 
for poverty levels of schools, as found in the prior year, within the quartile of schools 
with the highest poverty-level, Reading Recovery schools had significantly lower 
retention rates (.65%) than comparison schools (2.97%). However, in contrast to the 
previous year, 2001-2002 OELIs in the highest poverty schools had a substantially 
higher average retention rate (4.39%) than both comparison schools and schools with 
Reading Recovery interventions. Among schools in the upper-middle quartile (50th to 
75th percentile) of poverty, comparison schools’ retention rates (.54%) were higher 
than OEEI schools (.47%), but lower than Reading Recovery schools (.83%). That 
same 2001-2002 report found that retention rates for OEEI schools were lower than 
comparison schools’ retention rates in all but the highest poverty quartile. This find- 
ing, that more retention occurred in high-poverty OEEI schools vs. comparison 
schools, was considered a cause for some concern. 

While year-to-year differences in programs assessed and methodologies used make a 
single, definitive conclusion impossible, reports regarding retention have been relatively 
consistent across years. Reports have generally concluded that EEIGP funding was asso- 
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ciated with lower retention rates. And, Reading Recovery appears to have the strongest 
relationship with reduced retention in those schools with highest poverty levels. 

Student Outcomes - Special Education 
Referrals 

As noted in the 1998 report, researchers in special education have theorized that the 
rate of referral to special education assessment and eventual identification is a conse- 
quence of the “instructional tolerance” of a school (Gerber, 1988; Gerber & Semmel, 
1984). The theory reflects the legitimate constraints placed on a teacher given the 
number of students, heterogeneity of student ability, amount of instructional time, 
expertise, and resources. Often, realizing that limited time, expertise, or resources are 
available to help students at risk for reading failure, teachers refer students for special 
education assessment. Based on this rationale, two prior reports examined the relation- 
ship between grant initiatives and rates of special education referrals in ELIGP-funded 
and comparison schools. 

The early CEEP reports examined referral rates as a variable among funded and com- 
parison schools, although those comparisons were not done in the most recent analy- 
ses. In the 1997-1998 report, CEEP found that OEEI programs had significantly lower 
rates of referral for special education assessment than comparison schools. No statisti- 
cally significant differences were found between Reading Recovery and comparison 
schools. That report noted that this difference may be accounted for in part by the fact 
that OEEI programs tend to focus on a broader range of students within the general 
education classroom, whereas Reading Recovery is an individualized pullout instruc- 
tional strategy and serves fewer students but with more depth. 

A follow-up analysis was included in the 2000 report, “Progress in Early Eiteracy: 
Summary Evaluation of Indiana’s Early Eiteracy Intervention Grant Program 1997-98 
through 1999-2000 School Year.” In that analysis of data for the first three years of the 
program, trends for reported referral rates were found to be relatively flat for both 
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funded and comparison schools. While comparison schools tended to report higher 
referral rates across all three years, the differences were small and not statistically sig- 
nificant. The report also concluded that because the ELIGP funds target those schools 
with high needs for external support and those with elevated literacy needs, the full 
impact of the funding might not be seen in short-term studies, but continued monitor- 
ing was key to understanding the impact of the program on schools. 

Thus, while descriptive data provide some support for the hypothesis that OELI fund- 
ing would be associated with lower rates of special education referral, statistical tests 
of significance have been inconclusive. This measure of class-wide referrals may not 
be as appropriate for Reading Recovery programs as OEEI programs, since Reading 
Recovery is primarily a one-on-one program that provides intensive instruction to only 
a few students per class. 



Student Outcomes - Achievement 

CEEP reports in 2000-2001 and 2001-2002 addressed the relationship between grant 
initiatives and ISTEP-i- scores. This section summarizes those findings and additional 
results from a study using alternative achievement scores, all of which appear to pro- 
vide moderate evidence of achievement gains among EEIGP-funded schools. 

Two major methods of analyses have been used to examine the relationship between 
EEIGP interventions and ISTEP-i- passing rates. In some analyses, passing rates have 
been directly compared across funded and non-funded schools. The 2000 “Progress in 
Early Eiteracy: Summary Evaluation of Indiana's Early Eiteracy Intervention Grant 
Program 1997-98 through 1999-00 School Year” compared the impact of four of the 
major EEIGP programs — Reading Recovery, Success for All, Eiteracy Collaborative, 
and Eour Blocks — with comparison schools. The percentage of students passing read- 
ing achievement tests in Grade 3 was the outcome of interest. In this analysis, signifi- 
cant effects were not seen for Reading Recovery or Success for All. However, schools 
with Eiteracy Collaborative and Eour Blocks both demonstrated significantly higher 
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ISTEP+ reading test pass rates than comparison schools. Moderate effects (between .5 
and .7) were seen for Literacy Collaborative, and small effects (between .02 and .04) 
were seen for Four Blocks. 

The 2000-2001 report found that high-poverty schools with Reading Recovery had 
47.97% of their students pass the ISTEP-i- Grade 3 test, compared to a 45.06% pass 
rate for high-poverty comparison schools. Overall, OELI schools had the highest pass 
rates, with 64.92% pass rate in OELI schools with Grade 1-3 interventions, compared 
to 59.80% in Reading Recovery schools, and 62.91% in comparison schools. 

A subsequent report, in 2001-2002, found that at the start of the interventions, compar- 
ison schools had a higher average percent pass rate (64.98%) on Grade 3 ISTEP-i- lan- 
guage arts tests than did the group of schools receiving ELIGP funding (63.87%). 
They continued to have higher scores following that year's intervention, but the effect 
size of d = .02 suggested less of a difference between the two sets of scores than prior 
to the intervention. Schools with four ELIGP interventions (Four Blocks [75.17%], 
Early Success [80.67%], Literacy Collaborative [71.73%], and Waterford [76.06%]) 
had higher mean pass rates than the 70.58% rate found in the comparison schools. 
Reading Recovery and “other” early interventions were found to have passing rates of 
68.77% and 68.95%, respectively. That report pointed out that fall ISTEP-i- language 
arts pass rates for Grade 3 tests may not be as strong an indicator of the effects of 
ELIGP funding as retention rates because the tests are administered at the beginning of 
the academic year in Grade 3, therefore the students may not yet have received the 
ELIGP-funded services when the test is taken. Still, the data were sufficient for that 
report to conclude that “EIGP [sic] funding of interventions is associated with higher 
pass rates relative to comparison schools, especially in EIGP [sic] funded schools with 
highest need” (p. 17). 

In an alternate method of analysis, regression techniques have been used to assess 
which school and intervention characteristics are associated with elevated ISTEP-i- 
pass rates. A 2000 study reported on ISTEP-i- scores for the students in funded and 
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comparison schools from the first three years of the project. Two separate ordinary 
least squares (OLS) regression analyses were conducted to estimate the influence of 
funding type and of intervention type on ISTEP+ English/Language Arts passing rates 
within EEIGP and comparison schools. “Eunding type” refers to which EEIGP-funded 
literacy intervention was implemented by each school. Because not all literacy inter- 
ventions being used in schools are funded by EEIGP (for example, some schools 
implement Reading Recovery independent of EEIGP funding), a separate analysis was 
conducted using “intervention type” found within each school, regardless of funding 
source. In the first analysis, a model containing variables representing school charac- 
teristics, funding type, and literacy program features was found to explain 62.8% of 
variance in school passing rates. Three school characteristics were significant predic- 
tors of ISTEP-i- passing rates among schools. Having high average ISTEP-i- scores was 
associated with higher passing rates. Having a high percentage of poor students or 
minorities was associated with lower passing rates. And, one intervention variable was 
significant: having funding for Reading Recovery was negatively associated with high 
passing rates. However, in the second regression analysis, which considered “interven- 
tion type” rather than “funding type,” having a Reading Recovery program failed to be 
a significant predictor of ISTEP-t passing rate. These findings were interpreted to sup- 
port a view that Reading Recovery programs may be less effective in the first year of 
implementation than in following years, due to teacher training and learning curves. 
And, given this perspective, the findings of elevated ISTEP-t pass rates among stu- 
dents in the highest poverty schools were considered especially encouraging. 

Einally, a 2000 study assessed achievement gains among 618 students in two Indiana 
elementary schools that had received EEIGP funds. Pre- and post-tests using the Basic 
Academic Skills Samples (BASS) methodology were used to assess changes in basic 
literacy skill development. At “Elm Creek” Elementary (pseudonym), which 
expanded Reading Recovery and implemented Grade 2 Eiteracy Groups with grant 
funds, gains were seen both for Grade 1 and Grade 2 students in reading and writing. 
At “Sycamore Heights” Elementary (pseudonym), which implemented Eirst Steps, 
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mean gains were found for Grade 1, Grade 2, and Grade 3 students on both reading 
and writing, and were strongest among the Grade 3 group. While acknowledging limi- 
tations to the study, including a lack of a control group, the report presented evidence 
to suggest that the most impact was on students’ writing fluency, and that the focus on 
reading at Elm Creek led to the greatest impact on reading skills, while at Sycamore 
Heights, where the focus was on writing instruction, writing fluency benefited most. 



Comparing Programs 

While the primary purpose of the annual studies has been to assess the overall grant 
program rather than to compare different literacy approaches, the wealth of data gath- 
ered has provided valuable information on various literacy programs chosen and 
developed by grant recipients. CEEP reports have presented three types of compara- 
tive data: a) descriptive information on various literacy models and interventions as 
intended, b) frequency of instructional features as implemented, and c) statistical anal- 
ysis comparing programs on given outcome criteria. 

Programs as Intended 

Two CEEP publications have been prepared with the specific objective of helping edu- 
cators and administrators select programs based on each school's individual character- 
istics and planning assessment initiatives. “Improving Early Reading and Eiteracy: A 
Guide for Developing Research-Based Programs,” published in December 1999, pro- 
vided easy-to-read overviews of early reading and literacy intervention methods and 
programs. A framework for comparing interventions was developed for that publica- 
tion and used as the model for assessing and comparing program components in all 
subsequent CEEP evaluations. This model considers literacy outcomes of an interven- 
tion to be the result of a combination of the following: a) existing school theories, b) 
professional development features of the intervention, c) implemented philosophy of 
the intervention, d) parent involvement features of the intervention, e) classroom 
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instruction features of the intervention, and f) organizational/structural features of the 
intervention. This model has been the framework for all studies synthesized in the cur- 
rent report. 

A second resource guide, “Improving Early Reading: A Resource Guide for Elemen- 
tary Schools” was published by CEEP in Eebruary 2002. Drawing again from the 
CEEP framework for comparing early reading and literacy interventions, this report 
provides background on reading reforms and interventions, compares program fea- 
tures of 14 such programs, and offers suggestions for assessing early reading pro- 
grams. Readers are directed to these guides for thorough descriptions and analyses of 
early literacy models, and for comparisons of programs as they are intended to be 
implemented. 

Programs as Implemented 

Although the publications discussed above allow for descriptive comparison of inter- 
vention programs, statistical analyses provide more direct comparison of outcomes 
associated with adoption of such programs by Indiana schools. Surveys of teachers 
and principals have allowed CEEP to assess what characterizes actual implementation 
of the various models. 

Table 4 below presents the instructional features reported to be used most frequently 
by teachers in EEIGP-funded schools. The seven most common EEIGP interventions 
are included, with remaining interventions making up the “Other” category. Except for 
the “Other” category, the table presents features that were reported as being used more 
frequently in that intervention than in other EEIGP or comparison schools in either the 
2000-2001 or 2001-2002 study. Because interventions in the “Other” category varied 
between the two report periods, only features which were reported both in 2000-2001 
and 2001-2002 are included for that group. 
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TABLE 4. Table 4: Features frequently used by teachers In common ELIGP Interventions, compared with other- 

funded and comparison schools (based on 2000-2002 data). 



Reading Recovery 


* Pullout Instruction 

* Paired Reading 

* Written Observations 

* Reading Speciaiists 


Four Blocks 


* One-on-one Tutoriais 

* Ongoing Written Observations 

* Trade Books 

* Cooperative Learning 

* Paired Reading (Ciassroom) 

* Paired Reading (Ciassroom) 

* Emergent Speiiing 

* Basai Readers 

* Phonemic Awareness 

* Pattern Discrimination 

* Muiti-Sensory Phonics Activities 


Waterford 


• Independent Reading 

• Puii-out Instruction 

• Ongoing Written Observations 

• Paired Reading (Ciassroom) 

• Book Distribution 

• Take-home Reading Activities (neariy as often as 
comparison schoois) 

• Pattern Discrimination 

• Muiti-Sensory Phonics Activities 


Literacy Collaborative 


• Independent Reading 

• Puiiout Instruction 

• Smaii Groups, Teacher Directed 

• Ongoing Written Evaiuations 

• Trade Books 

• Creative Writing/Essays 

• Coiiaborative Teacher Pianning 

• Reading Speciaiist 

• Book Distribution 

• Chiid Centered/Deveiopmentai 
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TABLE 4. Table 4: Features frequently used by teachers In common ELIGP Interventions, compared with other- 

funded and comparison schools (based on 2000-2002 data). 



Early Success 


• Basal Readers (nearly as often as comparison 
schools) 

• Independent Reading 

• Systematic, Formative Evaiuation 

• Ongoing Written Observations 

• Paired Reading (Ciassroom and Parent) 

• Phonemic Awareness 

• Teacher Directed 

• Baianced Approach to Phoneme Instruction 

• Basai Readers 

• Smaii Groups, Teacher Directed 

• Emergent Speiiing 

• Reading Aioud 

• Paired Reading 


Success for All 


• Abiiity Grouping 

• Basai Readers 

• Cooperative Learning 

• Ongoing Written Observations 

• Paired Reading (Ciassroom and Parent) 

• Systematic Formative Evaiuation 

• Reading Driiis 

• Worksheets/Workbooks 


First Steps 


• Independent Reading 

• Smaii Groups, Teacher Directed 

• Trade Books 

• Creative Writing/Essays 

• Emergent Speiiing 

• Paired Reading (Ciassroom and Parent) 

• Parent Communication 

• Reading Aioud 

• Chiid Centered/Deveiopment 


Other 


• Independent Reading 

• One-on-One Tutoriai 

• Smaii Groups, Teacher Directed 

• Ongoing Written Observations 

• Creative Writing/Essays 

• Paired Reading 



In addition to pointing out features that discriminate funded interventions from each 
other, Table 4 serves to demonstrate one of the points frequently mentioned in CEEP 
reports-Reading Recovery programs do not necessarily include a great deal of unique 
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features compared with other interventions. Rather, that program, to some degree, may 
represent a balance of approaches common to comparison schools, but enhanced 
through one-to-one instruction and teacher training. 

Program Outcomes 

Because the primary purpose of the CEEP involvement in EEIGP funding has been to 
evaluate overall success of the program and to promote decisions and assessment at 
the school level, the degree to which prior studies have attempted to compare out- 
comes associated with particular programs or intervention features is limited. Compar- 
ing the success of specific programs across the life of the EEIGP program is complex, 
due to the number of variables involved, changes in the makeup of interventions each 
year, and various possible methods of defining and measuring success. However, 
taken together, findings shed light on which types of programs may be best suited for 
use in Indiana schools. This section summarizes select findings from prior CEEP 
reports attempting to compare outcomes of various EEIGP interventions, and con- 
cludes with a summary table of all CEEP findings on three student outcomes studied 
since 1997: special education referral rates, grade retention rates, and ISTEP-i- pass 
rates. 

Eirst, some comment on general approaches predictive of positive student outcomes is 
warranted. As noted previously in this report, CEEP findings have consistently cited 
advantages of balanced literacy approaches. Eor example, findings from the 2000 
report on the first three years of the program suggest that schools should seek a bal- 
ance between explicit and holistic approaches to early reading. Explicit approaches 
were positively associated with higher passing rates on achievement tests but also 
appeared to be associated with higher grade retention rates. In contrast, holistic 
approaches were negatively associated with retention rates, indicating that they were 
successful in moving students toward academic progress. That report concluded that 
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when selecting or designing an intervention, planning teams should “consider how the 
intervention would enable them to build an appropriate balance.” 

The first CEEP report attempting to summarize EEIGP data across several collection 
points was a 2000 report titled “Progress in Early Eiteracy: Summary Evaluation of 
Indiana's Early Eiteracy Intervention Grant Program 1997-98 through 1999-00 School 
Year.” That study compared Grade 3 student reading achievement tests for comparison 
schools and four of the major EEIGP programs: Reading Recovery, Success for All, 
Eiteracy Collaborative, and Eour Blocks. Eiteracy Collaborative had moderate effects 
(between .5 and .7). Eour Blocks had a small effect (between .02 and .04). Neither 
Reading Recovery nor Success for All had significant effects. Interpreted, schools 
with Eiteracy Collaborative and Eour Blocks both had significantly higher passing 
rates on the ISTEP-i- reading test than comparison schools, while neither Reading 
Recovery nor Success for All appeared to show similar elevated levels of passing 
rates. However, because poverty has been identified as a variable associated with liter- 
acy outcomes, it was controlled for in two additional analyses. Eirst, results were gen- 
erated for high-poverty and low-poverty schools separately. Eindings showed that 
effect sizes were found for all four programs in the high-poverty school group. There- 
fore, results in this section should be tempered by an understanding that additional 
variables, such as poverty, may mediate results. 

In that 2000 study, when a regression analysis controlled for pre-funding ISTEP-i- 
scores and percent of minority students. Success for All and Eiteracy Collaborative 
were associated with lower special education referrals, and Eour Blocks was associ- 
ated with higher special education referrals. Reading Recovery and Success for All 
were both associated with lower grade retention. The report concluded that these data 
provided stronger support for Eiteracy Collaborative, Success for All, and Reading 
Recovery than for Eour Blocks. Eurther findings with regard to poverty level are dis- 
cussed in a separate section of this report. 
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The 2000-2001 report concluded that most of the intervention models resulted in 
lower rates of retention and/or higher ISTEP-i- pass rates in language arts. Interven- 
tions consistent with this pattern included Reading Recovery, Literacy Collaborative, 
Four Blocks, First Steps, and Early Success. While these programs varied in their fea- 
tures and implementation, they all showed substantial signs of success. Success for All 
was found to be inconsistent in implementation and outcomes. The report also referred 
to literature suggesting a substantial start-up time for that program, leaving it to appear 
less advantageous than other programs evaluated. 

The 2001-2002 report found that too few schools used Success for All to be consid- 
ered a separate category. For other models mentioned in 2001 -Literacy Collaborative, 
Four Blocks, First Steps, and Early Success-consistent patterns of implementation and 
generally favorable outcomes were seen. That is, most of the interventions resulted in 
lower rates of retention and/or higher Grade 3 ISTEP-i- pass rates, even though they 
differed in their patterns of practice in terms of organizational and instructional fea- 
tures. Two findings pointed out as areas to be watched included implementation within 
Literacy Collaborative programs and Waterford Early Reading schools, which 
appeared to be less consistent than other OELI programs examined, and retention rates 
in Waterford Early Reading schools. But, that report concluded that “In Indiana, Read- 
ing Recovery, Literacy Collaborative, Four Blocks, Early Success, and Waterford were 
generally associated with improved outcomes and merit further implementation” (p. 
76). In the 2001-2002 site visits, the Waterford Early Reading program received mixed 
reviews. Overall, teachers and students found the program enjoyable and teachers 
agreed that it enhanced reading in the classroom. Concerns included maintenance 
expenses, the possibility of distraction, and difficulty of monitoring student computer 
use. 

In addition to examining success of the most common ELIGP interventions, CEEP 
reports have also addressed success rates of those interventions classified as “other” 
models, models that were not identified in a list of known interventions and/or were 
developed locally by those schools. Results from the 2000-2001 and 2001-2002 
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reports were somewhat discrepant, possibly due in part to the inclusion of a different 
set of interventions in that category in the second year. The 2000-2001 report con- 
cluded that the “other” interventions tended to include features that demonstrated a 
balance between independent, child-centered approaches and more systematic aspects. 
These schools cited emphasis on one feature — Code/Phonemes Taught Within Con- 
text — more heavily than did teachers in other types of schools. The report interpreted 
this as indicating a literature-rich environment, but with direct phonics deeply inte- 
grated into day-to-day practices. These schools were less likely to report higher or 
lower referral or retention rates compared with comparison schools, most likely 
because they were already high-performing schools, and “appear to be schools with 
good records in reading performance that are making an effort to get even better” (p. 
95). 

The 2001-2002 report also looked at the locally-developed “other” programs. In con- 
trast to 2000-2001, this year these models were not associated with appreciably better 
outcomes than comparison schools. A possible explanation was that the schools 
choosing these programs in 2001-2002 were not already high-performing schools 
compared to the 2000-2001 group. The 2002 report said that this group “struggled to 
produce appreciable student benefits.” The recommendation was that locally designed 
intervention funding decisions should be considered carefully. The decisions should be 
based on each school's previous level of success with literacy reform, and implement- 
ing research-based strategies may be preferable. 

Finally, Tables 5-8 summarize findings as reported in eight 1997-2002 CEEP studies 
comparing EEIGP and comparison schools. Three outcomes were included in those 
studies: special education referral rates, grade retention rates, and ISTEP-i- pass rates. 
Eiteracy programs assessed, outcomes of interest, and specific methods of analysis 
varied by year. These findings are not meant to substitute for time series or experimen- 
tal methodologies, which could provide the most definitive conclusions regarding the 
effectiveness of individual programs. Because pre-intervention levels were not avail- 
able in most cases, actual change within schools cannot be assessed, and therefore 
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conclusions for any given year should be viewed with prudence. Taken together, the 
cumulative results provide evidence supporting the performance of individual literacy 
programs funded by ELIGP. Readers are directed to the specific reports for further 
details of methodologies and findings within each study. 

In Tables 5-8, “NA” indicates that the report did not include analysis results for that 
outcome and group, and “NS” indicates that the group was included in analysis for that 
outcome, but statistical significance was not reached. Levels of statistical significance 
were .05, unless otherwise noted. In some cases, mean differences were presented but 
no statistical tests were performed, as indicated by “no statistical test of significance 
performed.” 

Table 5 presents results for those analyses which pooled all ELIGP-funded programs 
and presented findings which compared that group with non-funded comparison 
schools. In some cases these analyses crossed over several years of data, whereas other 
analyses were limited to a single year of data. 
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TABLE 5. Table 5. Results for ELIGP-Funded Programs Across Eight CEEP Evaluation Studies 



Program 


Referral Rates 


Retention Rates 


Achievement 


All ELIGP 


97-98 NA 


97-98 NA 


97-98 NA 


Schools 


98-99 Funded schools receiving 2 


98-99 Funded schools receiving 2 


98-99 NA 




years of funds had lower referral 


years of funds had lower retention 






rates than those with 1 year of 


rates than those with 1 year of 






funding. 


funding 






99-00 NS trends seen across 3 


99-00 Funded schools receiving 3 


99-00 NA 




years 


years of funding had lower reten- 
tion rates than comparison schools 
in 1998 and 1999, but not in 2000 








97-00 (1) NA 






97-00 (1) NA 


97-00 (2) NA 


97-00 (1) NA 




97-00 (2) NA 


97-00 (3) NA 


97-00 (2) NA 




97-00 (3) NA 


00-01 Funded schools had lower 


97-00 (3) NA 




00-01 NA 


retention rates than comparison 


00-01 Funded schools had lower 






schools 


ISTEP+ pass rates than compari- 
son schools, no statistical test of 
significance performed. Among 
highest poverty quartile schools, 
funded schools had higher ISTEP-r 
pass rates than comparison 
schools, no statistical test of signif- 
icance performed 






01-02 Funded schools had lower 






01-02 NA 


retention rates than comparison 


01-02 Funded schools had lower 






schools 


ISTEP-i- pass rates than compari- 
son schools; they also had lower 
rates before the funding, and the 
effect size for the difference 
between the groups was smaller 
after funding year 



Table 6 presents results for ELIGP-funded Reading Recovery programs from previous 
CEEP studies. Types of analysis included in those studies include comparison of 
means and regression procedures on special education referrals, grade retention, and 
ISTEP+ pass rates. 
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TABLE 6. 



Table 6. Results for ELIGP-Funded Reading Recovery Programs Across Eight CEEP Evaluation Studies. 



Program 


Referral Rates 


Retention Rates 


Achievement 


Reading 


97-98 NS difference with com- 


97-98 NS difference with com- 


97-98 NA 


Recovery 


parison schools 


parison schools 






98-99 NS difference with com- 


98-99 NS difference with com- 


98-99 NA 




parison schools 


parison schools 






99-00 NS difference with com- 


99-00 NS difference with com- 


99-00 NA 




parison schools 


parison schools 






97-00 (1) NS predictor of refer- 


97-00 (1) Predictor of lower 


97-00 (1) Predictor of higher 




rals, in regression analysis 


retention, in regression analysis 


ISTEP-r pass rates in high pov- 
erty schools 






97-00 (2) NS predictor, in regres- 






97-00 (2) Predictor of lower refer- 


sion analysis 


97-00 (2) NA 




rals, in regression analysis 


97-00 (3) NS predictor in first 






97-00 (3) NS predictor of refer- 


regression analysis, predictor of 


97-00 (3) Predictor of lower 




rals in either the first or second 


lower retention in second regres- 


ISTEP-r pass rates in first regres- 




regression analysis 


sion analysis 


sion analysis, NS predictor in 
second regression analysis 






00-01 Lower retention rates than 


00-01 Lower mean ISTEP-r pass 




00-01 NA 


comparison schools, no statist!- 


rates, no statistical test of signifi- 






cal test of significance performed 


cance performed. Among highest 
poverty quartile, higher mean 
pass rate than comparison 
schools, no statistical test of sig- 
nificance performed 






01-02 Lower retention rates than 
comparison schools, no statist!- 


01-02 Lower mean ISTEP-r 




01-02 N A 


cal test of significance per- 


passing rate than comparison 






formed. Among highest poverty 


schools, no statistical test of sig- 






quartile, lower mean retention 
than comparison schools, no sta- 
tistical test of significance per- 
formed 


nificance performed 



Table 7 presents results for ELIGP-funded programs other than Reading Reeovery, by 
intervention type. These were determined to be the most eommon intervention types 
funded by ELIGP, and were therefore examined individually, rather than in the 
“OEEI” eategory. This set of programs, drawn out for individual analysis, varied by 
year. Readers are referred to the original reports for eomplete details on whieh pro- 
grams were ineluded in eaeh analysis. 
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TABLE 7. Table 7. Results for ELIGP-Funded Programs Across Eight CEEP Evaluation Studies - By Type. 



Program 


Referral Rates 


Retention Rates 


Achievement 


Success for All 


97- 98 NA 

98- 99 NA 

99- 00 NA 


97- 98 NA 

98- 99 NA 

99- 00 NA 


97- 98 NA 

98- 99 NA 

99- 00 NA 




97-00 (1) Predictor of lower refer- 
ral rates, in regression analysis 

97-00 (2) NS predictor, in regres- 
sion analysis 

97-00 (3) Predictor of lower refer- 
ral rates, in regression analysis 

00-01 NA 


97-00(1) Predictor of lower reten- 
tion, in regression analysis 

97-00 (2) NS predictor, in regres- 
sion analysis 

97-00 (3) Predictor of lower reten- 
tion (weak.1 level) 


97-00 (1) Predictor of higher 
ISTEP-i- pass rates in high-poverty 
schools 

97-00 {2) NA 

97-00 (3) NS predictor, in regres- 
sion analysis 




01-02 NA 


00- 01 Higher mean retention than 
comparison schools, no statistical 
test of significance performed 

01- 02 N A 


00- 01 Lower mean ISTEP+ pass 
rates, no statistical test of signifi- 
cance performed 

01- 02 N A 


Literacy 

Collaborative 


97- 98 NA 

98- 99 NA 

99- 00 NA 


97- 98 NA 

98- 99 NA 

99- 00 NA 


97- 98 NA 

98- 99 NA 

99- 00 NA 




97-00 (1) Predictor of lower refer- 
ral rates, in regression analysis 


97-00 (1) NS predictor, in regres- 
sion analysis 


97-00 (1) Predictor of higher 
ISTEP-r pass rates in both low- 
and high-poverty schools 




97-00 (2) NS predictor of referral 
rates 


97-00 (2) Predictor of lower reten- 
tion rates 


97-00 (2) NA 




97-00 (3) Predictor of lower refer- 
ral rates, in regression analysis 


97-00 (3) NS predictor, in regres- 
sion analysis 


97-00 (3) NS predictor, in regres- 
sion analysis 




00-01 NA 


00-01 Lower retention rates than 
comparison schools, no statistical 
test of significance performed 


00-01 Higher mean ISTEP pass 
rate than comparison schools, no 
statistical test of significance per- 
formed 




01-02 NA 


01-02 lower mean retention than 
comparison schools, no statistical 
test of significance performed 


01-02 Higher mean passing rate 
than comparison schools, no sta- 
tistical test of significance per- 
formed 
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TABLE 7. 



Table 7. Results for ELIGP-Funded Programs Across Eight CEEP Evaluation Studies - By Type. 



Program 


Referral Rates 


Retention Rates 


Achievement 


First Steps 


97-98 NA 


97-98 NA 


97-98 NA 




98-99 NA 


98-99 NA 


98-99 NA 




99-00 NA 


99-00 NA 


99-00 NA 




97-00 (2) NA 


97-00 (1) NS predictor, in regres- 


97-00 (1) Predictor of lower 






sion analysis 


ISTEP+ pass rates, in regression 
analysis 




97-00 (1) Predictor of lower refer- 


97-00 (2) NS predictor, in regres- 


97-00 (2) NA 




ral rates, in regression analysis 


sion analysis 






97-00 (3) Prediotor of lower refer- 
ral rates, in regression analysis 


97-00 (3) NS predictor, in regres- 


97-00 (3) Predictor of lower 






sion analysis 


ISTEP+ pass rates (at .1 alpha 




00-01 NA 




level), in regression analysis 






00-01 Higher retention rates than 


00-01 Higher mean ISTEP-i- pass 






comparison schools, no statistical 


rates than comparison schools, no 






test of significance performed 


statistical test of significance per- 




01-02 NA 


01-02 NA 


formed 








01-02 NA 


Waterford 


97-98 NA 


97-98 NA 


97-98 NA 




98-99 NA 


98-99 NA 


98-99 NA 




99-00 NA 


99-00 NA 


99-00 NA 




97-00 (1) NA 


97-00 (1)NA 


97-00 (1)NA 




97-00 (2) NA 


97-00 (2) NA 


97-00 (2) NA 




97-00 (3) NA 


97-00 (3) NA 


97-00 (3) NA 




00-01 NA 


00-01 NA 


00-01 NA 




01-02 NA 


01-02 Higher retention rates than 


01-02 Higher mean ISTEP-i- pass 






comparison schools, no statistical 


rate than comparison schools, no 






test of significance performed 


statistical test of significance per- 
formed 
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TABLE 7. Table 7. Results for ELIGP-Funded Programs Across Eight CEEP Evaluation Studies - By Type. 



Program 


Referral Rates 


Retention Rates 


Achievement 


Four Blocks 


97-98 NA 


97-98 NA 


97-98 NA 




98-99 NA 


98-99 NA 


98-99 NA 




99-00 NA 


99-00 NA 


99-00 NA 




97-00 (1) Predictor of higher refer- 


97-00 (1) NS predictor, in regres- 


97-00 (1) Predictor of higher 




ral rates, in regression analysis 


sion analysis 


ISTEP-i- pass rates in both low- 
and high-poverty schools 




97-00 (2) NS predictor, in regres- 


97-00 (2) NS predictor, in regres- 


97-00 (2) NA 




sion analysis 


sion analysis 






97-00 (3) Predictor of higher refer- 


97-00 (3) NS predictor, in regres- 


97-00 (3) NS predictor, in regres- 




ral rates, in regression analysis 


sion analysis 


sion analysis 




00-01 NA 


00-01 Higher retention rates than 


00-01 Higher mean ISTEP+ pass 






comparison schools, no statistical 


rate than comparison schools, no 






test of significance performed 


statistical test of significance per- 
formed 




01-02 NA 


01-02 Lower retention rates than 


01-02 Higher mean ISTEP+ pass 






comparison schools, no statistical 


rates than comparison schools, no 






test of significance performed 


statistical test of significance per- 
formed 


Early 


97-98 NA 


97-98 NA 


97-98 NA 


Success 


98-99 NA 


98-99 NA 


98-99 NA 




99-00 NA 


99-00 NA 


99-00 NA 




97-00 (1) NA 


97-00 (1) NA 


97-00 (1) NA 




97-00 (2) NA 


97-00 (2) NA 


97-00 (2) NA 




97-00 (3) NA 


97-00 (3) NA 


97-00 (3) NA 




00-01 NA 


00-01 Lower mean retention than 


00-01 Higher mean ISTEP+ pass 






comparison schools, no statistical 


rate than comparison schools, no 






test of significance performed 


statistical test of significance per- 
formed 






01-02 Lower mean retention than 






01-02 NA 


comparison schools, no statistical 


01-02 Higher mean ISTEP+ pass 






test of significance performed 


rate than comparison schools, no 
statistical test of significance per- 
formed 



Table 8 presents findings for schools categorized as “Other Early Literacy Interven- 
tions (OELI).” There were schools funded by ELIGP, but for programs other than 
Reading Recovery or any of the other interventions assessed individually during a 
given year (shown in Table 7). The literacy programs included in this category varied 
from year to year, depending on which programs were funded and by which programs 
were examined individually. Readers are referred to the original reports for complete 
details on which programs were included in each analysis. 



Center for Evaluation and Education Poiicy 



33 of 48 



TABLE 8. Table 8. Results for ELIGP-Funded Programs Across Eight CEEP Evaluation Studies - Programs Coded 

“Other” (OELI). 



Program 


Referral Rates 


Retention Rates 


Achievement 


All OELI 
Schools* 


97-98 Significantly lower referral 
rates than comparison schools 


97-98 NS difference with com- 
parison schools 


97-98 NA 




98-99 NS difference with com- 
parison schools 


98-99 NS difference with com- 
parison schools 


98-99 NA 




99-00 NA 
97-00 (1) NA 
97-00 (2) NA 


99-00 NA 
97-00 (1) NA 
97-00 (2) NA 


99-00 NA 
97-00 (1)NA 
97-00 (2) NA 




97-00 (3) Predictor of lower refer- 
ral rates, In regression analysis 

00-01 NA 


97-00 (3) Predictor of lower 
retention, in regression analysis 

00-01 Lower retention rates than 
comparison schools, no statisti- 
cal test of significance performed 


97-00 (3) NS predictor in first 
regression analysis, NA in sec- 
ond regression analysis 

00-01 Higher mean ISTEP-r pass 
rates than comparison schools, 
no statistical test of significance 
performed. Also, among highest 
poverty quartile, higher mean 
pass rate, no statistical test of 
significance performed 




01-02 N A 


01-02 NA in first analysis. But 
among schools in highest pov- 
erty quartile, higher mean reten- 
tion rate than comparison 
schools. 


01-02 N A 



Table 9 presents a list of reports refereneed in Tables 5-8. 
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TABLE 9. Table 9 Reports Referenced in Tables 5-8. 



Years 

Referenced 


Publication 


97-98 


Manset, G., St. John, E., Simmons, A., Michael, R., Bardzell, J., Hodges, D., Jacob, S., 
& Gordon, D. (1999). Indiana's Early Literacy Intervention Grant Program: Implementa- 
tion Study for 1997-98. Bloomington, IN: Indiana Education Policy Center. 


98-99 


Manset, G., St. John, E., Simmons, A., Michael, R., Bardzell, J., Hodges, D., Jacob, S., 
& Gordon, D. (2000). Indiana's Early Literacy Intervention Grant Program: Implementa- 
tion Study for 1998-99. Bloomington, IN: Indiana Education Policy Center. 


99-00 


Manset, G., St. John, E., Simmons, A., Worthington, K., Chung, C-G., & Manoil, K. 
(2000). Indiana's Early Literacy Intervention Grant Program: Implementation Study for 
1999-2000. Bloomington, IN: Indiana Education Policy Center. 


97-00(1) 


Manset, G, St. John, E., & Simmons, A. (2000). Progress in Early Literacy: Summary 
Evaluation of Indiana's Early Literacy Intervention Grant Program 1997-98 Through 
1999-00 School Year. Bloomington, IN: Indiana Education Policy Center. 


97-00 (2) 


St. John, E.P., Manset, G., Chung, C., Simmons, A.B., Musoba, G.D., Manoil, K., & Wor- 
thington, K. (2000). Research-Based Reading Reforms: The Impact of State-Funded 
Interventions on Educational Outcomes in Urban Elementary Schools. Bloomington, IN: 
Indiana Education Policy Center. 


97-00 (3) 


St. John, E.P., Manset, G., Chung, C., Simmons, A.B., Musoba, G.D. (2000). Research- 
Based Reading Interventions: The Impact of Indiana's Early Literacy Grant Program. 
Bloomington, IN: Indiana Education Policy Center. 


00-01 


St. John, E.P., Michael, R.S., Chung, C., Simmons, A.B., Worthington, K. Manoil, K., & 
Loescher, S. (2001 ). Indiana's Early Intervention Grant Program Impact Study for 2000- 
01. Bloomington, IN: Indiana Education Policy Center. 


01-02 


Plucker, J.A., Hessing, J.J., Lim, W., St. John, E.P., Simmons, A.B., Patterson, A.P., & 
Dow, G.T. (2003). Indiana's Early Intervention Grant Program Impact Study for 2001- 
2002. Bloomington, IN: Indiana Education Policy Center. 



As demonstrated in Tables 5-8, of the individual ELIGP-funded interventions included 
in prior CEEP evaluations, Eiteracy Collaborative has most consistently demonstrated 
success on student outcomes. Reading Recovery, Success for All, and Eour Blocks 
have also demonstrated favorable results overall. Early Success shows promise, based 
on limited data collected on that intervention, and merits further attention. The Eirst 
Steps and Waterford programs are both associated with mixed results and should be 
examined further. 
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Meeting Needs of Students in Low-Income 
and Urban Schools 

Several CEEP reports have addressed the success of EEIGP with Indiana students con- 
sidered at highest need for literacy improvement. The intention of the EEIGP grant 
program was to improve reading skills of young students considered at risk for poor 
literacy outcomes. One group considered at greater risk of not learning to read consists 
of students who attend urban schools (Slavin, 1991; Snow, Burns, & Griffith, 1998). 
Also, a strong predictor of reading scores in general is the rate of poverty in a school. 
Thus, several CEEP reports have specifically attempted to assess EEIGP funding 
within these contexts. With regard to poverty, studies have reported on how funds have 
been allocated to high-poverty schools, and have collapsed data across poverty levels 
in order to assess effectiveness of programs specific to schools’ income status. One 
study looked specifically at EEIGP programming within urban schools in order to gen- 
erate conclusions most likely to be valid for that population. Brief summaries of these 
prior findings are summarized below. 

CEEP reports have consistently indicated that EEIGP funds were appropriated to 
schools with the most need, and have suggested a relationship between a school's pov- 
erty level and the type of EEIGP-funded program used by that school. The initial 
EEIGP assessment, 1997-1998, reported that schools that implemented Reading 
Recovery had lower SES status than OEEI or comparison schools. This was seen as an 
important consideration, particularly because it was thought to be a possible explana- 
tion for the limited measurable improvements over comparison and OEEI schools. 
Data from the 2000-2001 study also show that Reading Recovery schools had a mean 
of 34.5% of students eligible for free lunch, compared with 24.2% of students in Eiter- 
acy Collaborative schools, and 28.9% for other early interventions. 

Also, the 1998-1999 report found that schools in the second funding year had a greater 
percentage of students at risk for reading failure than comparison schools, according to 
income and achievement attainment indices. That is, schools that year had a signifi- 
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cantly greater percentage of students from low-income families in their schools and a 
greater percentage of students retained and referred for special education assessment 
than the randomly selected sample of comparison schools. They also had lower 
ISTEP-i- scores on the English/Language Arts scale. Based on this information, the 
report stated “that the program is meeting the goal of targeting those students at great- 
est risk for reading failure” (p. iv). 

In order to examine program effectiveness with regard to poverty level, reports 
between 2000 and 2002 used both comparative and regression methods of analysis to 
assess the relationship of income and program success within schools. 

The 2000 CEEP report, “Progress in Early Eiteracy: Summary Evaluation of Indiana’s 
Early Eiteracy Intervention Grant Program 1997-98 through 1999-00 School Year,” 
looked at Grade 3 reading achievement rates in low- and high-poverty schools imple- 
menting Reading Recovery, Success for All, Eiteracy Collaborative, and Eour Blocks. 
The low-poverty schools were in the quartile with the lowest percentages of students 
receiving free or reduced price lunch. High-poverty schools were in the quartile with 
high percentages of students receiving free or reduced price lunch. That study found 
that in low-poverty schools, only Eiteracy Collaborative and Eour Blocks showed sig- 
nificant gains in passing rates on ISTEP-i- reading test scores. However, within high- 
poverty schools, both Reading Recovery and Eour Blocks interventions demonstrated 
small effect sizes, and Success for All and Eiteracy Collaborative had effect sizes 
larger than found in low-poverty schools. Therefore, it appeared that any effect of each 
of the four programs was greatest in the high-poverty school group. Other findings 
included: 

• Reading Recovery was associated with reduced grade-level retention in high- 
poverty schools. 

• Success for All was associated with lower special education referral and grade- 
level retention in high-poverty schools. 

• Eiteracy Collaborative was associated with reduced lower special education 
referral in both high- and- low poverty schools. 
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• Four Blocks was associated with higher special education referral in both high- 
and low-poverty schools. 

A subsequent regression analysis on all schools controlled for pre-funding ISTEP-i- 
scores, percent of students receiving free or reduced cost lunch, and percent of minor- 
ity students in the school, and found poverty to be a consistent predictor of literacy- 
related outcomes. Controlling for the influence of poverty and other variables that 
influence literacy outcomes (pre-funding ISTEP-i- scores and percent minority), Read- 
ing Recovery, Success for All, and Eiteracy Collaborative were associated with 
improvement in literacy outcomes, leading that report to conclude that “clearly these 
programs merit further funding” (p. 19). The evidence for Eour Blocks was not consid- 
ered as compelling, since it appeared to be associated with ISTEP-i- passing scores but 
also with increased referrals. 

The 2000-2001 report presented strong evidence that EEIGP-funded interventions 
were most effective in schools with high poverty rates. In that study, ISTEP-i- Ean- 
guage Arts passing rates of Reading Recovery schools exceeded those of comparison 
schools only among schools in the quartile of highest poverty schools. OEEI school 
pass rates were higher than comparison rates in the highest two quartiles, but not the 
lowest two. Thus, that report concluded that “. . .findings suggest that the funded inter- 
ventions were most effective in schools with the greatest need” (p. 28). The report also 
found lower retention rates among Reading Recovery and OEEI schools than in com- 
parison schools, and that those differences were most notable in the schools with the 
highest levels of poverty. Although the comparison of means for retention rates did not 
achieve statistical significance, a clear trend in the direction of increased retention dif- 
ferences with increasing levels of poverty was noted. 

Again the following year, the 2002 CEEP report stressed the advantages of the EEIGP 
programs among schools serving with high poverty populations. Specifically, high- 
poverty Reading Recovery schools that year had 53% of their students pass the 
ISTEP-I- Grade 3 test prior to the funded intervention, compared to 56% for compari- 
son schools, but had 60% of students pass that test the following year (after EEIGP 
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funding was implemented), compared to 59% for comparison schools. That report 
stated, “Schools serving low-income students should be actively encouraged to partic- 
ipate in the Early Intervention Grant Program, especially given the record of success in 
schools with low-income students” (p. 76). 

Finally, a study published in 2000, “Research Based Reading Reforms: The Impact of 
State -Funded Interventions on Educational Outcomes in Urban Elementary Schools,” 
used regression analysis to examine three years of EEIGP data within urban schools. 
That study used a subpopulation of EEIGP and comparison schools located in urban 
school districts. In the average school in the sample, 44% of the students qualified for 
free or reduced price lunch and 31% were minorities. Among findings, the percentages 
of minority students and of students on free and reduced price lunch were significant 
and positively associated with retention rates. Having a teacher funded through Read- 
ing Recovery and having an OEEI intervention were associated with lower retention 
rates. The study found that explicit approaches did not relate to reductions in either 
special education referrals or retention, leading to the conclusion that some calls to 
place more emphasis on explicit instruction did not seem warranted based on the data. 
Rather, evidence suggested that having comprehensive approaches to early reading 
improvement was most beneficial for the urban school children, and that “having 
funded projects for both Reading Recovery and other literacy interventions (OEEI-1- 
3) [sic] improved educational progress related to early reading. Funding comprehen- 
sive programs aimed at improving early reading appears to make a difference for 
urban school children” (p. 26). The study also concluded that both funding and pro- 
gram maturity are important factors in the success of EEIGP interventions, particularly 
in the case of Reading Recovery. The study concluded “...there is strong evidence to 
support the idea that categorical funding for early reading interventions can improve 
educational outcomes in urban schools.” (p. 27). 
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Cost and Savings 

The first CEEP report, 1997-1998, estimated the state cost for training Reading Recov- 
ery teachers to be $917 per student served by those teachers. Costs for OELl projects 
were estimated at $162 per student served. This difference in cost reflects the differing 
focus of the programs, since Reading Recovery is a pull-out program that targets the 
lowest achieving 20% of students, where OELl programs typically involve the entire 
classroom of students in the intervention. 

Costs per student in the 1998-1999 report were estimated as approximately $677 per stu- 
dent for Reading Recovery students, and approximately $89 per student for OELl pro- 
grams. The following year, those costs were estimated at $257 for Reading Recovery 
program students and $157 for OELl program students. The costs of both programs 
were therefore said to be “considerably less than other common remedial options, such 
as grade retention and special education services” (p. 6). This was based on an estimate 
that each student retained in early primary grades cost the state and districts $4,387 that 
year, and that the average state cost for serving students identified as having a learning 
disability ranges from $1,522 - $2,577 a year. 

Based on a comparison of retention rates between funded and non-funded schools, the 
2000-2001 report concluded that, due to expenses associated with retaining students, 
Reading Recovery resulted in a savings of $1,200 for every 100 students, an amount 
equal to 55% of the average per student cost (p. 106), considering the cost of the inter- 
vention and savings due to decreased retention. This was based on an estimate that each 
student retained in early primary grades cost the state $5,399 for the 2000-2001 school 
year (Theobald & Michael, 2000). 

Thus, CEEP reports have concluded that in addition to weighing program gains in terms 
of student achievement, fiscal outcomes should be considered as well. Data demonstrat- 
ing reduced grade retention and special education referrals among ELlGP-funded 
schools indicate that the state may recover substantial portions of those grant funds 
when programs succeed. 
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Conclusions and Implications 

Whereas prior ELIGP studies conducted by CEEP have focused on one to three years 
of EEIGP data, this report spans the eight years of the program and seven years of 
evaluation findings in order to identify trends, themes, and conclusions that were not 
possible in the prior short-term reports. Major conclusions include the following: 

• The grant program has had a considerable impact on literacy education in Indi- 
ana. EEIGP has served more than 10,000 students per year and over 700 schools 
since its inception in 1997. Evaluation reports provide evidence that the grant 
program has resulted in direct improvements in areas of teacher expertise and 
collaboration, student achievement, lower grade retention, and in some cases a 
reduction in special education referrals. The program's indirect influence on 
statewide literacy practices over the past eight years, considering the degree to 
which professional development and collaboration may positively impact teach- 
ers and students in other grades and even other schools, is difficult to evaluate, 
although it should be considered in future evaluations. 

• Eunding and implementing research-based interventions appear to be associated 
with positive student outcomes. Providing categorical grants that encourage 
schools to select interventions that meet their local needs is an effective 
approach. An important part of program selection and implementation should be 
systematic evaluation within and across schools. Research serves to provide 
valuable information on cost-benefits, program components, and student out- 
comes related to various EEIGP-funded interventions. 

• Generally, the interventions funded by EEIGP do not represent drastic depar- 
tures from standard practice, but most seem to fit into prevailing Indiana literacy 
instructional practices. Interventions that take a balanced approach are most 
likely to complement Indiana's existing literacy education environment and 
appear to be most effective. EEIGP-funded programs tend to promote more stu- 
dent-centered literacy instruction than comparison schools, but within a balance 
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of skills-based and holistic instruction that appears to be associated with signifi- 
cant success. 

• The ELIGP program appears to have met the goal of reaching Indiana students 
with the greatest need for literacy improvement. CEEP reports have consistently 
concluded that EEIGP-funded programs, and Reading Recovery in particular, 
were associated with especially good outcomes in high-poverty schools. 
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Recommendations 

These conclusions lead to nine recommendations to the Indiana Department of Educa- 
tion (IDOE), in the areas of future research and evaluation, funding for Indiana early 
literacy initiatives, and providing support to schools in need of EEIGP resources: 

Research and Evaluation 

• Continue to conduct an annual survey of EEIGP and comparison schools with 
the sustained funding in the 2005-07 state budget. These activities serve to docu- 
ment how funding has been distributed, where those dollars have been spent, 
and what student gains can be attributed to the initiative. A variety of research 
methods have proven useful. Site visits and case studies provide in-depth and 
contextual information on the nature of specific interventions. Earge-scale anal- 
yses of the effects of EEIGP on standardized outcomes such as ISTEP-i- results, 
student retention, and special education referrals provide quantitative data that 
allow outcome comparisons across interventions, schools, and years. Euture 
evaluations should consider utilization of randomized experimental designs. 
Although this type of evaluation would take several years and considerable col- 
laborative efforts to achieve, the results would allow for rigorous evaluation 
information not available to date. This information would also be useful to any 
other state-funded or state- supported literacy interventions, such as the federal 
Reading Eirst program. 

• Continue to identify research-based programs and provide schools with that 
information in order to guide their program and funding choices. Provide 
schools with assistance in selecting or developing the most appropriate scientifi- 
cally-based reading literacy program for their needs. Eor example, evidence 
from EEIGP data suggests that balanced approaches to literacy instruction, 
incorporating features consistent with both phonics and whole language tradi- 
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tions, and including child- initiated as well as direct instruction elements, are 
most likely to result in success. 

• Encourage schools to review and refine their early reading and literacy pro- 
grams. More specifically, encourage more site-based action research to build a 
base of empirical data on program outcomes. It is especially important that new 
interventions be accompanied by systematic evaluations. 

Funding for Early Literacy Initiatives 

• Funding for packaged programs should be targeted on interventions with a 
proven record of success (i.e., evidence-based best practices), especially inter- 
ventions that focus on reading improvements in Grades 1-3. 

• Of ELIGP interventions assessed by CEEP, Eiteracy Collaborative appears espe- 
cially successful, demonstrating the most consistent positive results of programs 
examined. Reading Recovery, Success for All, and Four Blocks all demonstrated 
favorable results overall. Early Success shows promise, based on limited data 
collected on that intervention, and merits further attention. The First Steps and 
Waterford programs both received mixed results and should be examined fur- 
ther. 

• Given the Early Eiteracy Intervention Grant Program's record of success in 
schools with low-income students, all schools serving low-income students 
should be actively encouraged to participate in the program. Technical support 
for planning and applying for EEIGP funding should be made available to low- 
income schools that have not applied previously. 

Providing Support to Indiana Schoois 

• Increase emphasis on ongoing professional development in early reading and lit- 
eracy for elementary teachers. EEIGP evaluations demonstrate the importance 
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of professional development and collaboration in interventions seeking to 
improve early literacy outcomes. 

• Utilize local university support to enhance the implementation of early literacy 
intervention models. The IDOE should facilitate or encourage closer collabora- 
tion between schools and universities concerning ongoing professional develop- 
ment and evaluation that would benefit individual schools and the ELIGP grant 
program as a whole. 

• The IDOE should consider resuming efforts to provide grant workshops and 
expert guidance on reading research and literacy program implementation. 
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